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The evaluation of a program designed to assist principals recently 
appointed to their positions is the focus of this article. Researchers 
conducted a variety of qualitative assessments including focus group 
interviews, a review of training session feedback forms, participant 
reflective writing, and an assessment of training materials and program 
agendas. Researchers analyzed this data to look for aspects of participant 
knowledge, skills, and applications of program information. Participant 
feedback provided insights into program benefits and needed refinements. 
Article provides insights into assessment tools that people responsible for 
delivering principal support programs could consider to provide them 
with a more comprehensive examination of their program than traditional 
session feedback forms provide. Article cdso provides program 
recommendations that other program designers could consider to improve 
their existing principal support programs. Key Words: New Principals, 
Support Programs, Skill Development, Principal Professional 
Development, and Principal Leadership Development 


Introduction 

An ongoing educational theme over the last several years has been the importance 
of leadership development. Agencies such as colleges and universities, regional principal 
centers, state and national organizations, and other entities have developed programs to 
provide potential school leaders the support they need to be successful. 

One such program is being conducted in SW Virginia. The program is entitled the 
Recently Appointed Principals Program. This program is a collaboration between the 
Western Virginia Public Education Consortium and the Center for (COTA) at Virginia 
Polytechnic Institute and State University. 

This article highlights the evaluation of this program beyond just gathering 
participant perceptions at the end of a particular program session. The program directors 
asked us to construct and operate a comprehensive evaluation of the program. The 
readers of this article will learn how this evaluation was conducted, the results of the 
evaluation, and the recommendations we made to improve the delivery of the program. 
Our work could be the foundation for others charged with evaluating or organizing other 
similar leadership development programs. 

Content and Training Needs of School Leaders and Principals 

Today’s school leader faces many difficult challenges. Authors point to the 
increased scrutiny of schools, budget shortfalls, and student achievement challenges are 
among the many problems facing today’s school leader. As the challenges of society 
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have increased, the role of a principal or school leader has become increasingly difficult 
and complex. Crow (2006) outlined the increased accountability and complexity that 
makes the role of the principal difficult. He stated “...higher expectations for... principals 
in the area of instructional leadership... increased public scrutiny of public schools, and 
the promotion of privatization as a public policy agenda, have significantly changed the 
role of school principal'’ (p. 310). 

This high level of complexity can make it difficult for new principals to 
successfully acclimate into the job. The public looks to schools and their leaders to be 
able to work through the chaos and operate in a focused and effective manner. This can 
be a daunting and overwhelming task, especially for new principals or leaders. Walker 
and Qian (2006) state, “The dominant modern myth portrays the school principal as an 
underpaid workhorse tangling with the conflicting demands of instructional leadership, 
bureaucracy, official mandates and adverse interest groups...” (p. 298). 

In schools where problems are becoming more complex, there may be little time 
to prepare or plan for the succession of new principals into leadership positions. Some 
new school leaders are simply placed in a building and left to discover how to lead, 
satisfy the needs of the community, and support their teachers and students. Left on their 
own, some principals successfully figure out how to successfully navigate the 
environment but others flounder and even fail. New principals need support and guidance 
as they assume the role of school leader. 

Educational administration professors, universities, and leadership organizations 
typically provide support and assistance to aspiring school leaders and principals through 
principal preparation programs. Principal preparation programs are usually organized 
around standards. The Interstate School Leaders Licensure Consortium (ISLLC) 
standards are one type of standard used by principal preparation programs. Murphy and 
Shipman (1996) point out that the knowledge, disposition, and performance sections of 
the original ISLLC standards contain many technically-based skills. The original ISLLC 
standards also do not address strategies to enable principals to actually implement the 
standards. The 2008 revision of these standards does provide more insight into the 
implementation of the standards and the “human element” needed for success. New 
principals need to know the how as well as the what of leadership. 

Daresh and Playko (1994) identified three classifications of learning related to 
leadership skills important to new principals. They provide the following classifications 
from one of their studies of the topic “...beginning principals’ concerns were in three 
areas: problems with role clarification... limitations on technical expertise... and 
difficulties with socialization to the profession and to individual school systems...” (p. 
36). 

Daresh and Playko (1994) also found differences in the expressed needs of 
aspiring and practicing principals. These results are summarized in Ligure 1.1. 

Daresh and Plyko’s (1994) research indicates that aspiring principals express 
different needs than practicing principals. Walker and Carr-Stewart (2006) also confirm 
the importance of role clarification and socialization for new and practicing principals. 
Walker and Qian (2006) point out the importance of a focus on socialization and self- 
awareness in programs designed for new principals and how the development of these 
skills helps new principals as they work to define their values and in determining how 
they will work with their school community. Clearly the mastery of role clarification and 
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socialization is much more complicated than just following a written set of technical 
standards. 

Figure 1.1. Daresh and Playko’s (1994) Aspiring and Practicing Principals’ Ranking of 
Important Developmental Areas (Daresh & Playko, p. 39) 


Category 

Practicing Principals 
Ranking Related to 
Importance 

Aspiring Principals 
Ranking Related to 
Importance 

Technical Skills 
Socialization Skills 
Self-awareness (role 
clarification) Skills 

3 

1 

2 

1 

2 

3 


Roland Barth (1986) pointed out some of the challenges principals pose for staff 
developers when he referred to the fact that they “build up antibodies” (p. 156) toward 
efforts to training. Barth further contends that practicing principals are accustomed to 
leading rather than being led. 

Walker and Qian (2006) state, “...research holds strongly that beginning 
principals continue to learn through multiple pathways, such as reading, attending 
professional meetings, and conversing with professional friends...” (p. 303). Petzko 
(2004), Rich and Jackson (2005), and Browne-Ferrigno and Muth (2004) discuss the 
importance of professional networking and building meaning through discussion and 
dialog. 

This brief literature review was provided to help the reader understand the 
components of the program evaluated in this study and the perceived need of the 
organizers of the program to move beyond just gathering participant perceptions of each 
session. We were asked to construct an evaluation process that would assess several of 
the important learning processes such as multiple methods for learning, the impact of 
networking, the influence of reflection, and other techniques designed to be delivered 
within the Recently Appointed Principals Program. 

Description of the Leadership Development Program 

The Recently Appointed Administrators Program is the result of a collaborative 
effort between The Western Virginia Public Education Consortium (WVPEC) and the 
Center for Organizational and Technological Advancement (COTA) at Virginia 
Polytechnic Institute and State University. The program is facilitated by three 
coordinators coming from a variety of roles in public education. Four, 2 Vi day sessions 
are held each year, starting on Wednesday evenings and ending early in the afternoon on 
Friday. The program utilizes instruction from leaders in the field, interaction 
opportunities between participants, discussion of reading materials, and connections with 
a mentor. 
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Researcher Context 

The primary researcher for this project was Dr. John F Eller. Dr. Eller is a faculty 
member in the Leadership and Policy Studies Program at Virginia Polytechnic Institute 
and State University. He works in the National Capital Region in Northern Virginia 
working with Masters and doctoral students. The organizing groups responsible for 
delivering the Recently Appointed Principals Program in SW Virginia asked Eller to 
conduct the study as a disinterested third party. He has no connection or involvement 
with the program or any vested interest in the outcome of the evaluation. 

John F. Eller is interested in principal development programs and is the 
coordinator of the Principal Preparation Program in the Northern Virginia Region. He is 
interested in studying leadership development and learning about participant reflections 
based on their experiences. His intentions related to this study were to gather and share 
participant perceptions of the program and make accurate recommendations for its 
improvement. 

Eller also worked with another professor in his department who provided 
technical support and checked Eller’s coding and assessment processes. 

Program Evaluation Methods 

We conducted the study of the Recently Appointed Administrators Program from 
May through October of 2007. The initial program review was conducted using 16 
members of the 2005-2006 program cohort. Program data were gathered using a 
questionnaire, two different focus group sessions (each conducted with about Vi of the 
participant group), program session agendas, training materials, participant session 
feedback forms, and participant reflective writing samples. These data were initially 
sorted into the categories of knowledge, skills, and applications. Since program 
coordinators initially promised this participant group information related to their 
participation would be kept confidential, researchers were not allowed to contact 
principals directly beyond the focus group meeting or visit schools to gather additional 
data. 

Data were analyzed related to the major themes of the program and in relation to 
the knowledge, skills, and applications the participants gained from the sessions. The 
study/evaluation was also designed to examine the relationship between the “planned” 
curriculum, the “delivered” curriculum, and the “applied” curriculum. This examination 
enabled the researchers to study what participants were able to actually use from their 
experiences in the program back at their school sites. 

Data were also plotted related to Daresh and Playko’s (1994) three major learning 
areas related to the needs of practicing principals. These are: 

■ Role clarification (who they are as principals and how they should use 
their new power) 

■ Technical expertise (how to do what they were supposed to do 
according to their job description) 

■ Socialization issues (fitting into a particular setting/assignment) 
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Specific Participant Feedback Regarding Program Strengths 

Focus group comments and session feedback from participants provided specific 
information related to the strengths of the program: 

■ The collegial and networking aspects of the program were identified as a 
major area of strength by participants. In addition to meeting and talking 
with other principals, participants reported benefit in discovering 
colleagues that struggled with similar problems and who could help 
generate solutions to issues they faced. 

■ Program guest speakers and presenters were seen as another area of 
strength. Comments highlighted presenter preparation, credibility, clarity, 
and a focus on practical information during sessions. Some participants 
expressed concern that several presenters went over their allotted time and 
caused other presenters to omit important content. 

■ Program participants openly expressed their gratitude for their 
participation in the program and said they had personally thanked their 
superintendent for supporting them in the program. 

■ Many comments about the positive learning atmosphere established and 
maintained in the program and how it positively impacted their 
professional learning. 

■ Specific technical content was provided in the sessions and was related to 
issues that these new principals face such as teacher evaluation, data 
driven decision making, school safety and emergency preparedness, and 
compliance with state and national regulations. 

■ Group role playing and interaction sessions related to content were seen as 
helpful and necessary for success. Participants commented that role 
playing helped them implement ideas from the seminars back at their 
schools. 

■ Participants reported that areas that related to understanding and working 
with staff members, clarifying their role as a leader, attaining and 
maintaining balance, and delegation and involvement were among the 
most important but unexpected outcomes of the program. They reported 
that this knowledge helped them back at their buildings even more than 
some of the more technical aspects of the program. 

Participant Comments Classified According Need Areas 

Participant written comments on questionnaires administered prior to focus group 
meetings were classified in relation to Deresh and Playko’s (1994) need areas. The results 
of this sorting is included in Figure 1.2. 
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Figure 1.2. Focus Group Written Comments Based on the Deresh’s (1994) Need and 
Potential Use Levels 



Socialization 

Role Clarification 

Technical 

Content Knowledge 

6 

6 

7 

Skills 

2 

6 

18 

Applications 

17 

7 

17 

Totals 

25 

19 

41 


Focus group verbal comments were also classified in relation to need areas. The 
results of this classification is included in Figure 1.3. 

Figure 1.3. Focus Group Verbal Comments based on the Deresh’s (1994) Need and 
Potential Use Levels 



Socialization 

Role Clarification 

Technical 

Content Knowledge 

6 

6 

8 

Skills 

3 

4 

10 

Applications 

3 

5 

8 

Totals 

12 

15 

26 


Training materials and session agenda topics were analyzed according to need 
areas. The results of this analysis are included in Figure 1.4. 

Figure 1.4. Program Content Themes Plotted in Relation to Socialization, Role 
Clarification, and Technical Skills 


Socialization 

Role Clarification 

Technical 

Building Relationships 
Building healthy 
relationships 

Leadership (site specific) (2 
subtopics) 

Standards 

ISLLC standards for 
principals 

Leadership (4 subtopics) 
Maintaining Balance (2 
subtopics) 

Legal Issues (2 subtopics) 
IDEA/special education (4 
subtopics) 

Data (9 subtopics) 

Teacher Evaluation (3 
subtopics) 

Public Relations (1 
subtopic) 

3 Main topics 
2 Subtopics 

4 Main topics 
6 Subtopics 

5 Main topics 
19 Subtopics 


Participant Recommendations for Program Refinement 

Participants offered their recommendations regarding areas of refinement for 
future programming. 


Increase Informal Networking and Problem Solving Opportunities: 
Participants said they would have liked to have less time structured for 
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them and more unstructured time scheduled to be used for discussion, 
informal problem solving, and networking. 

■ Use of Current Technology: Participants addressed the need for more 
use of technology both at formal sessions and as a way to 
communicate and network between sessions(laptops, Internet, E-mail) 

■ Clear Presenter Guidance and Expectations: Participants recommended 
that session presenters stay within their assigned times. This would 
allow all presenters to have the time they need in order to fully 
develop their topic areas. 

■ Focus on Role Clarification and Socialization Content Early in the 
Program: Participants outlined the need for introducing information 
regarding balance (role clarification) and understanding their staff and 
community (socialization) into the program early to help them use the 
information in their schools earlier in the process. 

Future assessment plans and evaluation procedure refinements. 

Yearly program evaluations to assess program effectiveness and the match 
between its objectives and the needs of the principals it serves are planned by program 
coordinators. In order to assist with this ongoing longitudinal assessment we made the 
following program recommendations: 

■ Program and Individual Session Agenda Design: Sessions seemed to be 
content/topic driven rather than outcome driven. We recommended that 
clear, outcome -based objectives based on the general categories of 
knowledge, skills, and applications be developed for the program. Once 
these categories are in place, program planners should use these objectives 
to drive the development of materials, content presentations, agendas, and 
program measurement tools. Participants should be made aware of the 
particular outcomes to be highlighted at the beginning of each session. 

These outcomes could also be classified in relation to whether they 
provide information related to technical skills, role clarification, or 
socialization. 

■ Material and Presentation Guidance: As we examined the materials used 
by presenters, it was obvious that the quality and length of the materials 
provided by presenters was inconsistent in the program. Some presenters 
provided extensive but seemingly irrelevant material for short sessions 
while other presenters who were allotted more agenda time provided brief 
materials. Researchers recommended that presenters and resource people 
be provided clear guidelines to guide the development of their session 
content and learning materials. These guidelines should include material 
length and type, a focus on session outcomes, and application assignments 
to increase the chance of implementation success at the school level. In 
addition to providing a road map for implementation, this guidance would 
also raise the expectation that the knowledge and skills provided in the 
program need to be applied by participants. 
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■ Consistent Session Feedback Forms:Three different variations of the 
evaluation form were used in the program. Researchers recommended that 
consistent feedback forms should be used in all future program sessions. 

Higher Level Feedback: Many of the questions on the evaluation forms 
were either general in nature or related to perceptions of the learning 
environment. Future feedback forms should be redesigned to gather 
information related to the application of program information at 
participant schools. Feedback could also be gathered related to the 
categories of socialization, role clarification, and technical applications. 

By making the feedback forms more specific, program staff will be able to 
better understand what participants claim they are applying from their 
experience and by making the feedback forms more specific; program 
staff will be able to better understand what participants claim they are 
applying from their experiences in the program. 

Study Limitations 

This article highlights the work of two researchers conducting an initial program 
evaluation for a support program for recently appointed principals. Readers of this article 
should remember that the results of this study are subject to several limitations. First, this 
is one study of an isolated program in a specific geographic region of the country. The 
issues examined here may not be the same issues other programs will face. Also, the 
researchers selected to examine this program and sorting information into the 
classifications of knowledge, skills, and applications. These categories made sense in 
light of the program we examined so the data is sorted in this manner. These categories 
may not make sense for other programs wishing to examine their structure. We also 
selected the work of Daresh and Playko (1994) to use as a secondary classification 
structure. Again, these characteristics may not be appropriate for other programs 
considering an evaluation. 

Recommendations for Planners of Leadership Support Programs 

An ongoing area of need in the future will be the induction and support of new 
school leaders to the profession. While any program that is designed will provide 
principals with some level of support, there were some important lessons learned from 
this program evaluation that may be of benefit to others who are planning programs 
designed to support new or existing leaders. The following recommendations are offered: 

■ Organize programs around the themes of knowledge, skills, and 
applications-By organizing program content and activities around the 
themes of knowledge, skills, and applications, program leaders provide 
a way for participants to sort what they are learning. Program 
organizers should also consider using the concepts of role clarification, 
socialization, and technical skills as an organizing framework for their 
participants. 
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■ Understand and utilize the importance of informal networking 
opportunities- Program leaders should consider setting aside open and 
unstructured time to allow participants to informally network and learn 
from each other. 

■ Feedback related to the application of program skills should be 
utilized-Principal support program organizers should consider 
developing session feedback forms that move beyond gathering 
information related to the “feel” of the workshop or the quality of the 
food and facilities and ask participants to provide information related 
to their ability to use and apply program content. 

■ Presenter and Handout Guidance-By providing clear and specific 
guidance to resource people and presenters and asking them to include 
implementation and follow-up strategies in their training materials, 
program organizers increase the chances that principals will be able to 
understand and apply what they are learning in the sessions at their 
schools. 

The support and development of principals or school leaders in the field are 
rewarding and challenging. If programs are designed to take into account the learning and 
processing needs of these professionals, they can be extremely valuable. We hope that 
what was learned in the initial evaluation of the Southwestern Virginia Recently 
Appointed Principals Program provides ideas and strategies to program organizers and 
others in the field to consider as they are designing and implementing principal staff 
development programs. 
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